HDDS-14041. Add metrics to track Snapshot RocksDB space and SST File stats. by sadanand48 · Pull Request #9406 · apache/ozone

sadanand48 · 2025-12-01T13:50:23Z

What changes were proposed in this pull request?

Add metrics to track Snapshot RocksDB space and SST File stats

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-14041

How was this patch tested?

Tried out on docker

  "beans" : [ {
    "name" : "Hadoop:service=OzoneManager,name=OMSnapshotDirectoryMetrics",
    "modelerType" : "OMSnapshotDirectoryMetrics",
    "tag.Context" : "Snapshot Directory Metrics",
    "tag.Hostname" : "b8f9c29c90ba",
    "DbSnapshotsDirSize" : 1162516,
    "TotalSstFilesCount" : 36,
    "NumSnapshots" : 3
  } ]
}

…stats.

Copilot

Pull request overview

This PR adds comprehensive metrics tracking for Ozone Manager's snapshot RocksDB directories, monitoring disk space usage and SST file counts. The implementation introduces a new metrics class that periodically collects statistics both at the aggregate level (total snapshots directory size and SST file count) and per-checkpoint directory level, with configurable update intervals.

Key Changes

Introduces OMSnapshotDirectoryMetrics class with asynchronous metrics collection using a Timer-based scheduler
Adds configuration property ozone.om.snapshot.directory.metrics.update.interval (default: 5 minutes) to control update frequency
Integrates metrics lifecycle into OzoneManager's start/restart/stop methods

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 15 comments.

Show a summary per file

File	Description
`OMSnapshotDirectoryMetrics.java`	New metrics class implementing periodic collection of snapshot directory size and SST file statistics with both aggregate and per-checkpoint metrics
`OzoneManager.java`	Integrates snapshot directory metrics into OM lifecycle by starting metrics collection on start/restart and stopping on shutdown
`OMMetrics.java`	Adds snapshot directory metrics management methods and field to hold the metrics instance
`OMConfigKeys.java`	Defines configuration key and default value for metrics update interval
`ozone-default.xml`	Adds configuration property documentation for the metrics update interval setting

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

jojochuang · 2025-12-09T00:01:52Z

@sadanand48 also I updated the jira release note. PTAL

jojochuang

lgtm. Just want to see what copilot says but it seems to get stuck.

swamirishi

Thanks for the patch @sadanand48 have some nitpicky review comments on the code modularity and design of this. Let us make this change to beautify the code a bit so that it would be easier to add more metrics later on.

swamirishi · 2025-12-12T22:48:57Z

+ */
+@InterfaceAudience.Private
+@Metrics(about = "OM Snapshot Directory Metrics", context = OzoneConsts.OZONE)
+public final class OMSnapshotDirectoryMetrics implements MetricsSource {


Instead of having inner static classes. I would suggest to create snapshot.metrics package have these package private classes. It would make the code more readable.

swamirishi

Have a few doubts left comments inline

swamirishi · 2025-12-12T23:13:21Z

+   */
+  private long calculateDirSizeAccountingForHardlinks(File directory)
+      throws IOException {
+    Set<Object> visitedInodes = new HashSet<>();


Shouldn't this be a globalSet ? We can make this class variable and clear at the end of each run to reuse

the directory here is the db.snapshots/checkpointState dir that has all checkpoint dirs so we should be good

swamirishi · 2025-12-12T23:19:45Z

+
+    // Add per-checkpoint-directory metrics from cached map
+    Map<String, CheckpointMetrics> currentMetrics = checkpointMetricsMap;
+    for (Map.Entry<String, CheckpointMetrics> entry : currentMetrics.entrySet()) {


currentMetrics is a HashMap order would be indeterministic. We should use a treeMap to ensure this

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copilot

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 9 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

github-actions · 2026-01-10T00:05:35Z

This PR has been marked as stale due to 21 days of inactivity. Please comment or remove the stale label to keep it open. Otherwise, it will be automatically closed in 7 days.

github-actions · 2026-01-17T00:05:39Z

Thank you for your contribution. This PR is being closed due to inactivity. If needed, feel free to reopen it.

jojochuang

lgtm. merging it.

Sadanand Shenoy added 3 commits December 1, 2025 18:42

HDDS-14041. Add metrics to track Snapshot RocksDB space and SST File …

e5bec71

…stats.

fix checkstyle

56109a1

fix test

5d8c2ea

sadanand48 added the snapshot https://issues.apache.org/jira/browse/HDDS-6517 label Dec 1, 2025

jojochuang requested review from Copilot, jojochuang, smengcl and swamirishi December 1, 2025 17:14

Copilot started reviewing on behalf of jojochuang December 1, 2025 17:15 View session

Copilot finished reviewing on behalf of jojochuang December 1, 2025 17:17

Copilot AI reviewed Dec 1, 2025

View reviewed changes

address copilot comments

9baa25a

jojochuang reviewed Dec 8, 2025

View reviewed changes

Comment thread ...ne-manager/src/main/java/org/apache/hadoop/ozone/om/snapshot/OMSnapshotDirectoryMetrics.java Outdated

jojochuang reviewed Dec 8, 2025

View reviewed changes

Comment thread ...ne-manager/src/main/java/org/apache/hadoop/ozone/om/snapshot/OMSnapshotDirectoryMetrics.java Outdated

address comments

c8cff4a

sadanand48 marked this pull request as ready for review December 10, 2025 08:33

jojochuang requested a review from Copilot December 12, 2025 20:39

Copilot started reviewing on behalf of jojochuang December 12, 2025 20:40 View session

jojochuang approved these changes Dec 12, 2025

View reviewed changes

swamirishi requested changes Dec 12, 2025

View reviewed changes

smengcl reviewed Dec 13, 2025

View reviewed changes

Comment thread hadoop-hdds/common/src/main/resources/ozone-default.xml

smengcl reviewed Dec 13, 2025

View reviewed changes

Comment thread ...ne-manager/src/main/java/org/apache/hadoop/ozone/om/snapshot/OMSnapshotDirectoryMetrics.java Outdated

Copilot AI reviewed Dec 13, 2025

View reviewed changes

Comment thread ...ne-manager/src/main/java/org/apache/hadoop/ozone/om/snapshot/OMSnapshotDirectoryMetrics.java Outdated

Comment thread ...ne-manager/src/main/java/org/apache/hadoop/ozone/om/snapshot/OMSnapshotDirectoryMetrics.java Outdated

Sadanand Shenoy added 3 commits December 18, 2025 01:51

address comments

818e77d

address comments

a6efe09

pmd fix

39dd489

jojochuang requested a review from swamirishi December 18, 2025 19:19

jojochuang requested a review from smengcl December 18, 2025 19:19

smengcl requested a review from Copilot December 18, 2025 19:49

Copilot started reviewing on behalf of smengcl December 18, 2025 19:49 View session

Copilot AI reviewed Dec 18, 2025

View reviewed changes

address copilot comments

1177fbf

github-actions Bot added the stale label Jan 10, 2026

github-actions Bot closed this Jan 17, 2026

jojochuang reopened this Apr 2, 2026

github-actions Bot removed the stale label Apr 3, 2026

jojochuang approved these changes Apr 6, 2026

View reviewed changes

jojochuang merged commit 5c53d2e into apache:master Apr 6, 2026
44 checks passed

Conversation

sadanand48 commented Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

What is the link to the Apache JIRA

How was this patch tested?

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Key Changes

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jojochuang commented Dec 9, 2025

Uh oh!

jojochuang left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

swamirishi left a comment

Choose a reason for hiding this comment

Uh oh!

swamirishi Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

swamirishi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

swamirishi Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

sadanand48 Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

swamirishi Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jan 10, 2026

Uh oh!

github-actions Bot commented Jan 17, 2026

Uh oh!

sadanand48 commented Dec 1, 2025 •

edited

Loading

jojochuang left a comment •

edited

Loading

swamirishi Dec 12, 2025 •

edited

Loading